Experiencing Various Massively Parallel Architectures and Programming Models for Data-Intensive Applications

With the future workloads predicted to feature petascale data sets, we developed a course on exploring various massively parallel architectures and programming models for data-intensive applications. The course covers the architectures of three processors, AMD/ATI RV670 Streaming graphics processors, Nvidia G80 graphics processors, and Cell Broadband Engines, and their latest programming support. The course starts with the single-program multiple-data (SPMD) programming model on ATI/Nvidia graphics processors and extends to the multiple-program multiple-data (MPMD) model using Cell processors. In this paper, we report our experience in developing & teaching this course and present some interesting insights learnt from exploring these different architectures. The results from the students’ program experiences are promising: the SPMD model is easy to grasp and with a sound understanding of architectural features, significant performance gains can be achieved through various program optimizations. For example, one optimized matrix multiplication algorithm has been developed, which outperforms the carefully tuned library code (Nvidia CUBLAS) by 49%.